Skip to content

Decode the decimal min/max of a flat node's stats - #740

Merged
dfa1 merged 1 commit into
mainfrom
fix/decimal-stats-710
Nov 12, 2026
Merged

dfa1 merged 1 commit into
mainfrom
fix/decimal-stats-710

Conversation

@dfa1

@dfa1 dfa1 commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Fixes #710

Rust writes a decimal stat bound as the little-endian two's-complement bytes of its unscaled value (1, 2, 4, 8, 16 or 32 bytes; ScalarValue::to_proto / bytes_from_proto at 0.93.0). The parser read every bytes_value as text, so a decimal bound never ordered and a filtered scan could not prune on it.

ArrayStatsParser.parse(fbs, dtype) now builds a BigDecimal with the column's scale. An invalid width is rejected as Rust does, which readFlatStats already reads as no stats.

No vortex-jni cross-check: 0.93 writes no min/max on a decimal root node (byte parts or plain), so the path is unreachable from files it writes. Unit tests pin Rust's encoding at every width, negative values and negative scale.

🤖 Generated with Claude Code

@dfa1
dfa1 force-pushed the fix/decimal-stats-710 branch from af8bad9 to 7d26daf Compare November 12, 2026 21:21
Rust writes a decimal bound as the little-endian two's-complement bytes of its
unscaled value (1, 2, 4, 8, 16 or 32 bytes, ScalarValue::to_proto). The parser
read every bytes_value as text, so a decimal bound never ordered and a filtered
scan could not prune on it. ArrayStatsParser.parse takes the column dtype and
builds a BigDecimal from the dtype's scale; an invalid width is rejected as
Rust's bytes_from_proto does, which readFlatStats already reads as no stats.

No vortex-jni cross-check: 0.93 writes no min/max on a decimal root node
(byte parts or plain), so the path is only reachable from a file Rust writes
with those stats; the unit tests pin Rust's encoding at every width.

Fixes #710

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dfa1
dfa1 force-pushed the fix/decimal-stats-710 branch from 7d26daf to 8acb012 Compare November 12, 2026 21:23
@dfa1
dfa1 merged commit 9f0539a into main Nov 12, 2026
6 checks passed
@dfa1
dfa1 deleted the fix/decimal-stats-710 branch November 18, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decode the decimal scalar of a Rust-written flat node's min/max stats

1 participant